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Boosting of Image Denoising Algorithms 

Yaniv Romano^ and Michael Elad^ 


Abstract. In this paper we propose a generic recursive algorithm for improving image denoising methods. Given 
the initial denoised image, we suggest repeating the following ”SOS” procedure: (i) (S)trengthen 
the signal by adding the previous denoised image to the degraded input image, (ii) (O)perate the 
denoising method on the strengthened image, and (iii) (S)ubtract the previous denoised image from 
the restored signal-strengthened outcome. The convergence of this process is studied for the K- 
SVD image denoising and related algorithms. Still in the context of K-SVD image denoising, we 
introduce an interesting interpretation of the SOS algorithm as a technique for closing the gap 
between the local patch-modeling and the global restoration task, thereby leading to improved 
performance. In a quest for the theoretical origin of the SOS algorithm, we provide a graph- 
based interpretation of our method, where the SOS recursive update effectively minimizes a penalty 
function that aims to denoise the image, while being regularized by the graph Laplacian. We 
demonstrate the SOS boosting algorithm for several leading denoising methods (K-SVD, NLM, 
BM3D, and EPLL), showing tendency to further improve denoising performance. 

Key words. Image restoration, denoising, boosting, sparse representation, K-SVD, graph Laplacian, graph 
theory, regularization. 
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1. Introduction. Image denoising is a fundamental restoration problem. Consider a given 
measurement image y G obtained from the clean signal x G by a contamination of 

the form 

(1.1) y = x + v, 

where v G is a zero-mean additive noise that is independent with respect to x. Note 

that X and y are held in the above equation as column vectors after lexicographic ordering. 
A solution to this inverse problem is an approximation x of the unknown clean image x. 

Plenty of sophisticated algorithms have been developed in order to estimate the original 
image content, the NLM [6], K-SVD [15], BM3D [13], EPLL [53], and others [51, 9, 35, 26, 
46, 36]. These algorithms rely on powerful image models/priors, where sparse represeutatious 
[5, 14] aud processiug of local patches [27] have become two promiueut iugredieuts. 

Despite the effectiveuess of the above deuoisiug algorithms, improved results cau be ob- 
taiued by applyiug a boostiug techuique (see [46, 10, 31] for more details). There are several 
such techuiques that were proposed over the years, e.g. ’’twiciug” [49], Bregmau iteratious 
[32], / 2 -boostiug [8], SAIF [46] aud more (e.g. [36]). These algorithms are closely related aud 
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share in common the use of the residuals (also known as the ’’method-noise” [6]) in order 
to improve the estimates. The residual is defined as the difference between the noisy image 
and its denoised version. Naturally, the residual contains signal leftovers due to imperfect 
denoising (together with noise). 

For example, motivated by this observation, the idea behind the twicing technique [49] is 
to extract these leftovers by denoising the residual, and then add them back to the estimated 
image. This can be expressed as [10] 

(1.2) x"+i=x'= + /(y-x'=), 

where the operator / (•) represents the denoising algorithm and is the iteration denoised 
image. The initialization is done by setting = 0. 

Using the concept of Bregman distance [4] in the context of total-variation denoising [39], 
Osher et al. [32] suggest exploiting the residual by 

( k 

y + ^(y-V) 

where the recursive function is initialized by setting x^ = 0. Note that if the denoising 
algorithm / (•) can be represented as a linear (data-independent) matrix. Equations (1.2) and 
(1.3) coincide [10]. Furthermore, for these two boosting techniques, it has been shown [46] 
that as k increases, the estimate x^ returns to the noisy image y. 

Motivated by the above-mentioned algorithms, our earlier work [36] improves the K-SVD 
[15], NLM [6] and the first-stage of the BM3D [13] by applying an iterative boosting algorithm 
that extracts the ’’stolen” image content from the method-noise image. The improvement is 
achieved by adding the extracted content back to the initial denoised result. The work in [36] 
suggests representing the signal leftovers of the method-noise patches using the same basis/ 
support that was chosen for the representation of the corresponding clean patch in the initial 
denoising stage. As an example, in the context of the K-SVD, the supports are sets of atoms 
that participate in the representation the noisy patches. 

However, in addition to signal leftovers that reside in the residual image, there are noise 
leftovers that are found in the denoised image. Driven by this observation, SAIF [46] offers 
a general framework for improving spatial domain denoising algorithms. Their algorithm 
controls the denoising strength locally by filtering iteratively the image patches. Per each 
patch, it chooses automatically the improvement mechanism: twicing or diffusion, and the 
number of iterations to apply. The diffusion [31] is a boosting technique that suggests repeating 
applications of the same denoising filter, thereby removing the noise leftovers that rely in the 
previous estimate (sometimes also sacrificing some of the high-frequencies of the signal). 

In this paper we propose a generic recursive function that treats the denoising method as 
a ” black-box” and has the ability to push it forward to improve its performance. Differently 
from the above methods, instead of adding the residual (which mostly contains noise) back to 
the noisy image, or filtering the previous estimate over and over again (which could lead to 
over-smoothing), we suggest strengthening the signal by leveraging on the availability of the 
denoised image. More specifically, given an initial estimation of the cleaned image, improved 
results can be achieved by repeating iteratively the following SOS procedure: 
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1. Strengthen the signal by adding the previous denoised image to the noisy input image. 

2. Operate the denoising method on the strengthened image. 

3. Subtraet the previous denoised image from the restored signal-strengthened outcome. 
The core equation that describes this procedure can be written in the following form: 



(1.4) 


where = 0 . As we show hereafter, a performance improvement is achieved since the signal- 
strengthened image can be denoised more effectively compared to the noisy input image, due 
to the improved Signal to Noise Ratio (SNR). 

The convergence of the proposed algorithm is studied in this paper by formulating the 
linear part of the denoising method and assessing the iterative system’s matrix properties. 
In this work we put special emphasis on the K-SVD and describe the resulting denoising 
matrix and the corresponding convergence properties related to it. The work by Milanfar [31] 
shows that most existing denoising algorithms (e.g. NLM [6], Bilateral filter [47], LARK [11]) 
can be represented as a row-stochastic positive definite matrices. In this context, our analysis 
suggests that for most denoising algorithms, the proposed SOS boosting method is guaranteed 
to converge. Therefore, we get a straightforward stopping criterion. 

In addition, we introduce an interesting interpretation of the SOS boosting algorithm, 
related to a major shortcoming of patch-based methods: the gap between the local patch¬ 
processing and the global need for a whole restored image. In general, patch-based methods 
(i) break the image into overlapping patches, (ii) restore each patch (local processing), and 
(iii) reconstruct the image by aggregating the overlapping patches (the global need). The 
aggregation is usually done by averaging the overlapping patches. The proposed SOS boosting 
is related to a different algorithm that aims to narrow the local-global gap mentioned above 
[37]. Per each patch, this algorithm defines the difference between the local (intermediate) 
result and the patch from the global outcome as a ’’disagreement”. Since each patch is 
processed independently, such disagreement naturally exists. 

Interestingly, in the context of the K-SVD image denoising, the SOS algorithm is equivalent 
to repeating the following steps (see [37] and Section 6.2 for more details): (i) compute the 
disagreement per patch, (ii) subtract the disagreement from the degraded input patches, (iii) 
apply the restoration algorithm on these patches, and (iv) reconstruct the image. Therefore, 
the proposed algorithm encourages the overlapping patches to share their local information, 
thus reducing the gap between the local patch-processing and the global restoration task. 

The above should remind the reader of the EPLL framework [53], which also addresses the 
local-global gap. EPLL encourages the patches of the final image (i.e. after patch-averaging) 
to comply with the local prior. In EPLL, given a local patch model, the algorithm alternates 
between denoising the previous result according to the local prior, followed by an image 
reconstruction step (patch-averaging). Several local priors can use this paradigm - Gaussian 
Mixture Model (GMM) is suggested in the original paper [53]. Similarly, EPLL with sparse 
and redundant representation modeling has been recently proposed in [43]. EPLL bares some 
resemblance to diffusion methods [31], as it amounts to iterated denoising with a diminishing 
variance setup, in order to avoid an over-smoothed outcome. In practice, at each diffusion 
step, the authors of [53, 43] empirically estimate the noise that resides in (which is neither 
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Gaussian nor independent of x^). In contrast, in our scheme, setting this parameter is trivial 
- the noise level of y + is nearly a, regardless of the iteration number. 

In the context of image denoising, several works (e.g. [16, 2, 23, 24, 21]) suggest repre¬ 
senting an image as a weighted graph, where the weights measure the similarity between the 
pixels/patches. Since the graph Laplacian describes the structure of the underlying signal, it 
can be used as an adaptive regularizer, as done in the above-mentioned methods. Put differ¬ 
ently, the graph Laplacian preserves the geometry of the image by promoting similar pixels 
to remain similar, thus achieving an effective denoising performance. It turns out that the 
steady-state outcome of the SOS minimizes a cost function that involves the graph Laplacian 
as a regularizer, providing another justification for the success of our method. Furthermore, 
inffuenced by the SOS mechanism, we offer novel iterative algorithms that minimize the graph 
Laplacian cost functions that are defined in [16, 2, 23]. Similarly to the SOS, the proposed it¬ 
erative algorithms treat the denoiser as a ” black-box” and operate on the strengthened image, 
without an explicit construction of the weighted graph. 

This paper is organized as follows: In Section 2 we provide brief background material on 
sparse representation and dictionary learning, with a special attention to the K-SVD denoising 
and its matrix form. In Section 3 we introduce our novel SOS boosting algorithm, study 
its convergence, and generalize it by introducing two parameters that govern the steady- 
state outcome, the requirements for convergence and the rate-of-convergence. In Section 4 
we discuss the relation between the SOS boosting and the local-global gap. In Section 5 
we provide a graph-based analysis to the steady-state outcome of the SOS, and offer novel 
recursive algorithms for related graph Laplacian methods. Experiments are brought in Section 
6, showing a meaningful improvement of the K-SVD image denoising, and similar boosting 
for other methods - the NLM, BM3D, and EPLL. Conclusions and future research directions 
are drawn in Section 7. 

2. K-SVD Image Denoising Revisited. We bring the following discussion on sparse rep¬ 
resentations and specifically the K-SVD image denoising algorithm, because its matrix inter¬ 
pretation will serve hereafter as a benchmark in the convergence analysis. 

2.1. Sparse Representation 8i K-SVD Denoising. The sparse-land modeling [5, 14] as¬ 
sumes that a given signal x ^ (in this context, the signal x is not necessarily an image) 
can be well represented as x = Da, where D G is a dictionary composed of m > n 

atoms as its columns, and a G is a sparse vector, i.e, has a few non-zero coefficients. For a 
noisy signal y = x -\-v^ we seek a representation a that approximates x up to an error bound, 
which is proportional to the amount of noise in v. This is an NP-hard problem that can be 
expressed as 

(2.1) d = min||a||Q s.t. ||Da — ^||2 < 

a 

where ||a||o counts the non-zero coefficients in a, and the constant e is an error bound. There 
are many efficient sparse-coding algorithms that approximate the solution of Equation (2.1), 
such as OMP [33], BP [12], and others [14, 48]. 

The above discussion assumes that D is known and fixed. A line of work (e.g. [17, 42, 1]) 
shows that adapting the dictionary to the input signal results in a sparser representation. 
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In the case of denoising, under an error constraint, since the dictionary is adapted to the 
image content, the subspace that the noisy signal is projected onto is of smaller dimension, 
compared to the case of a fixed dictionary. This leads to a stronger noise reduction, i.e, better 
restoration. Given a set of measurements {yi}fLi^ ^ typical dictionary learning process [ 1 , 17] 
is formulated as 


( 2 . 2 ) 




N 

i=l 


N 


min > 7 j||Q;j||o + ||DQ!j 




where D and are the resulting dictionary and representations, respectively. The scalars 

7 j are signal dependent, so as to comply with a set of constraints of the form ||Daj — yjUl < e^. 

Due to computational demands, adapting a dictionary to large signals (images in our case) 
is impractical. Therefore, a treatment of an image is done by breaking it into overlapping 
patches (e.g. of size 8 x 8 ). Then, each patch is restored according to the sparsity-inspired 
prior. More specifically, the K-SVD image denoising algorithm [15] divides the noisy image 
into ^/nx^/n fully overlapping patches, then processes them locally by performing iterations of 
sparse-coding (using OMP) and dictionary learning as described in Equation (2.2). Finally, the 
global denoised image is obtained by returning the cleaned patches to their original locations, 
followed by an averaging with the input noisy image. The above procedure approximates the 
solution of 


(2.3) 


N 




mm 


/^l|x -y||2 + 


|2 

l2) 


i=l 


where x G is the resulting denoised image, N is the number of patches, and G 
is a matrix that extracts the patch from the image. The first term in Equation (2.3) 
demands a proximity between the noisy and denoised images. The second term demands that 
each patch R^x is represented sparsely up to an error bound, with respect to a dictionary D. 
As to the coefficients 7 ^, those are spatially dependent, and set as explained in Equation ( 2 . 2 ). 

2.2. K-SVD Image Denoising: A Matrix Formulation. The K-SVD image denoising can 
be divided into non-linear and linear parts. The former is composed of preparation steps that 
include the support determination within the sparse-coding and the dictionary update, while 
the outcome of the latter is the actual image-adaptive filter that cleans the noisy image. The 
matrix formulation of the K-SVD denoising represents its linear part, assuming the availability 
of the non-linear computations. At this stage we should note that the following formulation 
is given as a background to the theoretical analysis that follows, and it is not necessary when 
using the proposed SOS boosting in practice. 

Sparse-coding determines per each noisy patch R^y a small set of atoms D^. that partic¬ 
ipate in its representation. Following the last step of the OMP [33], given D^., the represen¬ 
tation^ ai of the clean patch is obtained by solving 

(2.4) ai ^ m.m\\'Ds^z-Kiy\\l, 


^We abuse notations here as ai refers hereafter only to the non-zero part of the representation, being a 
vector of length \si \ ^ m. 
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which has a closed-form solution 

(2.5) 

Given the clean patch is obtained by applying the inverse transform from the represen¬ 
tation to the signal/ patch space, i.e., 

( 2 . 6 ) Pi = 

Notice that although the computation of Si is non-linear, the clean patch is obtained by 
filtering its noisy version, R^y, with a linear, image-adaptive, symmetric and normalized filter. 

Following Equation (2.3) and given all p^ = the globally denoised image x is 

obtained by minimizing 


N 


(2.7) 


X = min /i||x - y||| + W ||pj - Rjx|| 2 . 


i=\ 


This is a quadratic expression that has a closed-form solution of the form 
(2.8) x= + l^/xy + ^Rfpij 

= D-^Ky 
= Wy, 


where I G is the identity matrix. The term is a diagonal matrix that 

counts the appearances of each pixel (e.g. 64 for patches of size 8x8) and /il originates from 
the averaging with the noisy image y. The matrix R^ returns a clean patch p^ to its original 
location in the global image. The matrix W G is the resulting filter matrix formulation 

of the linear part of the K-SVD image denoising. In the context of graph theory, D and K are 
called the degree and similarity matrices, respectively (see Section 5 for more information). 

A series of works [46, 31, 45] studies the algebraic properties of such formulations for 
several image denoising algorithms (NLM [6], Bilateral filter [47], Kernel Regression [11]), for 
which the filter-matrix is non-symmetric and row-stochastic matrix. Thus, this matrix has 
real and positive eigenvalues in the range of [0,1], and the largest eigenvalue is unique and 
equals to 1, with a corresponding eigenvector [1,1,...,!]^ [40, 22]. In the K-SVD case, and 
under the assumption of periodic boundary condition^, the properties of the resulting matrix 
somewhat different, and are given in the following theorem. 

^See Appendix A for an explanation on this requirement. 
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Theorem 2.1. The resulting matrix W has the following properties: 

1. Symmetrie W = and thus all eigenvalues are real. 

2. Positive definite W >- 0^ and thus all eigenvalues are strietly positive. 

3. Minimal eigenvalue ofW satisfy A^^^(W) > where n is the pateh size. 

4 . Doubly stoehastie, in the sense ofWl = = 1. Note that W may have negative 

entries, whieh violates the elassie definition of row- or eolumn- stoehastieity. 

5. The above implies that 1 is an eigenvalue eorresponding to the eigenveetor 1. 

6. The speetral radius o/W equals to 1, i.e, ||W ||2 = 1. 

7. The above implies that maximal eigenvalue satisfy Xmax (W) = 1. 

8. The speetral radius ||W — I ||2 < < 1. 

Appendix B provides a proof for these claims. 

For the denoising algorithms studied in [46, 31, 45], the matrix W is not symmetric nor 
positive definite, however it can be approximated as such using the Sinkhorn procedure [31]. 
In the context of the K-SVD, as describe in Appendix A, W can become symmetric by a 
proper treatment of the boundaries (essentially performing cyclic processing of the patches). 

To conclude, the discussion above shows that the K-SVD is a member in a large family of 
denoising algorithms that can be represented as matrices [31]. We will use this formulation 
in order to study the convergence of the proposed SOS boosting and for demonstrating the 
local-global interpretation. 

3. SOS Boosting. In this section we describe the proposed algorithm, study its conver¬ 
gence, and generalize this algorithm by introducing two parameters that govern its steady-state 
outcome, the requirements for convergence and its rate. 

3.1. SOS Boosting - The Core Idea. Leading image/patch priors are able to effectively 
distinguish the signal content from the noise. However, an emphasis of the signal over the 
noise could help the prior to better identify the image content, thereby leading to better 
denoising performance. As an example, the sparsity-based K-SVD could choose atoms that 
better fit the underlying signal. Similarly, the NLM, which cleans a noisy patch by applying 
a weighted average with its spatial neighbors, could determine better weights. This is the 
key idea behind the proposed SOS boosting algorithm, which exploits the previous estimation 
in order to enhance the underlying signal. In addition, the proposed algorithm treats the 
denoiser as a ”black-box”, thus it is easy to use and becomes applicable to a wide range of 
denoising methods. 

As mentioned in Section 1, the first class of boosting algorithms (twicing [49] or its variants 
[32, 8, 36]) suggest extracting the ’’stolen” content from the method-noise image, with the 
risk of returning noise back to the denoised image, together with the extracted information. 
On the other hand, the second class of boosting methods (diffusion [31] or EPLL [53, 43]) 
aim at removing the noise that resides in the estimated image, with the risk of obtaining an 
over-smoothed result (this depends on the number of iterations or the denoiser parameters 
at each iteration). As a consequence, these two classes of boosting algorithms are somewhat 
lacking as they address only one kind of leftovers [46] - the one that reside in the method-noise 
or the other which is found in the denoised image. Also, these methods may result in under- 
or over- smoothed version of the noisy image. 

Adopting a different perspective, we suggest strengthening the signal by adding the clean 
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image to the noisy input y, and then operating the denoising algorithm on the strengthened 
result. Differently from diffusion filtering, as the estimated part of the signal is emphasized, 
there is no loss of signal content that has not been estimated correctly (due to the availability 
of y). Differently from twicing, we hardly increase the noise level (under the assumption that 
the energy of the noise which resides in the clean image is small). Finally, a subtraction of 
from the outcome should be done in order to obtain a faithful denoised result. This procedure 
is formulated in Equation (1.4): 

= / (^y + -x^ 

where x^ = 0. 

The SOS boosting obtains improved denoising performance due to higher SNR of the 
signal-strengthened image, compared to the noisy input. In order to demonstrate this, let us 
denote 


(3.1) X = X + v^, 

where is the error that resides in the outcome x, containing both noise residuals and 
signal errors. Assuming that the denoising algorithm is effective, and x has an improved SNR 
compared to y, this means that 

(3.2) 

implying 

(3.3) ||v^|| = <^||v||, where 5 <C 1. 



Thus, referring now to the addition y + x, its SNR satisfies 

l|2x|p 


(3.4) 


SNR^(y+ x) = 


|v + VrIP 


> 


4||x|| 


IviP + 2||v|| llv^ll + llvr IP 


In the above we used the Cauchy-Shwartz inequality. Using (3.3) we get 

4||x|p 


(3.5) 


Since 5 1, we have that 


SNR^(y+ x) > 


(1 + (^)^||v|p 


(1 + ^)' 


rSNR2(y). 


(3.6) SNR(y + x) > SNR(y), 
where in the ideal case (5 = 0), the relation becomes 

(3.7) SNR(y + x) = 2.SNR(y). 
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3.2. Convergence Analysis. Studying the convergence of the SOS boosting is done by 
leveraging the linear matrix formulation of the denoising algorithm. The error of the SOS 
recursive function 

(3.8) efc = x^-x*, 
is defined as the difference between the estimate, 

(3.9) x^ = Wfc (^y + - x^-\ 

and the outcome that is obtained after a large number iterations, 

(3.10) x*=W*(y + x*)-x*, 

where is a filter matrix, which is equivalent to applying / (•) on the signal-strengthened 
image. Substituting Equations (3.9) and (3.10) into Equation (3.8) lead to 

(3.11) Ck (y + - x^'-i - (W* (y + V) - V) 

= (Wfc - W,) y + Wfcx^-i - Wfcx* + Wfcx* - W,x* - (x^-^ - x*) 

= (Wfc - W,) y + WkCk-i + (Wfc - W,) V - ek-i 
= (Wfc - I) + (Wk - W*) (y + V), 


where we use the recursive connection e^-i = x^“^ — x*. We should note that the non-linearity 
of /(y-hx^“^) = Wkiy + is neglected in the above derivation by allowing an operation 

of the form ^/Vkiy + x^~^) = 

In the following convergence analysis we shall assume a fixed filter-matrix W that operates 
on the signal-strengthened image along the whole SOS-steps, i.e., W = This comes 

up in practice after applying the SOS boosting for a large number of iterations (as explained in 
the context of Figure 1). In this case, the above-mentioned abuse of the non-linearity becomes 
correct, and thus the convergence analysis is valid. 

Theorem 3.1. Assume that W = and that the speetral radius of the transition 

matrix ||W — I ||2 = 7 < 1. The error e^ eonverges exponentially, i.e., \\ek\\2 < ll^olb • 7 ^ ^ 0 
for k ^ 00 . Thus, the SOS reeursive funetion is guaranteed to eonverge. 

Proof. By assigning = Wx,, the second term (Wk — Wx=) (y -h x*) in Equation (3.11) 
vanishes, thus 

(3.12) ek = {W-l)ek_i 

= (W - I)^ eo 

where eo = — x* = —x* is a constant vector. Using matrix-norm inequalities we get 
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where we use ||W — 1||2 = 7 . As a result, \\ek \\2 is bounded by 7 ^ • ||eo ||2 and approaches zero 
for A: ^ oc when 7 < 1. ■ 

As such, the SOS boosting is guaranteed to converge for a wide range of denoising al¬ 
gorithms - the ones that can be formulated/approximated such that W — I is convergent, 
e.g., the K-SVD [15], NLM [ 6 ], Bilateral filter [47] and LARK [11]. In the next sub-section 
we soften the convergence requirements and intensify its properties, along with a practical 
demonstration. 

3.3. Parametrization. We generalize the SOS boosting algorithm by introducing two pa¬ 
rameters that modify the steady-state outcome, the requirements for convergence (the eigen¬ 
values range) and its rate. Starting with the first parameter, p, which controls the signal 
emphasis, the formulation proposed is: 

(3.14) = / (y + px'^) - px^ 

where a large value of p implies a strong emphasis of the underlying signal. Assigning = 
— X* replacing / (•) with a fixed filter-matrix lead to 

(3.15) X* = W* (y -h px*) - px*, 
which implies a steady-state result 

(3.16) X* = (I + p(I-W,))-^W,y. 

This is the new steady-state outcome, obtained only if the SOS boosting converges. We should 
note that this outcome also minimizes a cost function that involves the graph Laplacian as 
a regularizer (see Section 5for further details). The conditions for convergence are studied 
hereafter. 

The second parameter, r, modifies the eigenvalues of the error’s transition matrix, thereby 
leading to a faster convergence and relaxing the requirement that only /(•) with eigenvalues 
between 0 to 1 is guaranteed to converge. We introduce this parameter in such a way that 
it will not affect that steady-state outcome (at least as far as the linear approximation is 
concerned). We start with the steady-state relation 

(3.17) X* = / (y + p±*) - p±*. 

We multiply both sides by r and add the term x* — x* to the RHS, 

(3.18) Tx* — rf {y -\- px*) — rpx* -h x* — x*. 

Thus, the same x* solving (3.17) will also solve (3.18) and thus the steady-state is not affected. 
Rearranging this equality leads to 

(3.19) V =r/(y + pV)-(rp + r-l)x*. 

As a result, the proposed generalized SOS boosting is given by 

(3.20) x^+^ = r/ (y + px'^) - (rp + r - l)x^. 
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It is important to note that although r does not affect x* explicitly, it may modify the 
estimates over the iterations. Due to the adaptivity of /(•) to its input, such modifications 
may eventually affect the steady-state outcome. 

Studying the convergence of Equation (3.20) is done in the same way that was taken in 
Section 3.2. Starting with the error computation, expressed by 

(3.21) 

Cfc =X^ - X* 

=rWk (y + px^“^) - (rp + r - l)x^“^ - (rW* (y + px*) - (rp + r - l)x*) 

=r {Wk - W*) y + rpWfcX^'-i - rpW^V + rpW^V - rpW*V - (rp + r - l)efc_i 
= (rpWfc - (rp + r - 1)1) Bk-i + r (W^ - W*) (y + pV). 

Next, following Theorem 3.1, and assuming W = W/^ = W*, we get that the condition for 
convergence is: 

(3.22) Vi (/)(t, p, Xi) = IrpXi - (rp + r) + 1| < 1, 

where {Xi}fLi and {0 (t, p, Xi)}fLi are the eigenvalues of W and the error’s transition matrix, 
respectively. In order to achieve the fastest convergence, we seek for the parameter r* that 
minimizes 


(3.23) T* = min max (/)(t, p, AD s.t. Vi (/)(t, p, AD < 1. 

r l<i<N 

Given r*, the rate-of-convergence is governed by 


(3.24) 


7* = max </>(r*,p,Ai). 

l<i<N 


Appendix D provides the following closed-form solution for Equation (3.23), 

2 


(3.25) 


T* = 


2(p “h 1) p{Xfyiin T ^max) 

along with optimal convergence rate, 

pi^^max ^min) 


(3.26) 


* 

7 = 


2(p + 1) — p{\rnin + A max) 


In the context of the K-SVD image denoising [15], Figure 1 demonstrates the properties 
of the generalized SOS recursive function for the image House, corrupted by zero-mean Gaus¬ 
sian noise with a = 25. Each K-SVD operation includes 5 iterations of sparse-coding and 
dictionary-update with noise level of d = 1.05a. We repeat these operations for 100 SOS- 
steps and set x* = x^^^. In the following experiment, the denoised images are the outcome of 
= W(y -h x^) — x^, where W is held fixed as W = Wioo, initializing with xq = 0. 

According to Theorem 2.1 and based on the original K-SVD parameters (n = 8 , m = 256, 
p = 1.02), we get Xmin ^ 0.015 and Xmax — 1- These are leading to r* = 0.67 (see Equation 
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(a) logiodle^lh) (b) 


Figure 1. Illustration of the generalized SOS reeursive funetion properties: (a) eonvergenee and (b) PSNR 
improvement. These graphs are generated by operating the K-SVD denoising [15] on noisy (a — 25) House 
image. 

(3.25)). Figure 1(a) plots the logarithm of \\ek \\2 for [ti,T 2 ,T 3 ] = As can be seen, 

the error norm decreases linearly, and bounded by c • 7^^, where [71,72, 73] = [0.66, 0.33, 0.98]. 
The fastest convergence is obtained for T 2 = r* = 0.67 with 72 = 7 * = 0.33. While the slowest 
one is obtained for T3 = 1 with 73 = 0.98. 

Figure 1(b) demonstrates the PSNR improvement (the higher the better) as a function 
of the SOS-step. As can be seen, faster convergence of \\ek \\2 translates well into faster 
improvement of the final image. The SOS boosting achieves PSNR of 32.78dB, offering an 
impressive improvement over the original K-SVD algorithm that obtains 31.8dB. 

4. Local-Global Interpretation. As described in Section 1, there is a stubborn gap be¬ 
tween the local processing of image patches and the global need (creating a final image by 
aggregating the patches). Consider a denoising scenario based on overlapping patches (e.g. 
[15, 51]): At the local processing stage, each patch is denoised independently^ without any 
influence from neighboring patches. Then, a global stage merges these outcomes by plainly 
averaging the local denoising results. 

Inspired by game-theory ideas, in particular the ’’consensus and sharing” optimization 
problem [3], we introduce an interesting local-global interpretation to the above-proposed 
SOS boosting algorithm. A game theoretical terminology of a patch-based processing can 
be viewed as the following: There are several agents, where each one of them adjusts its 
local variable to minimize its individual cost (in our case - representing the noisy patch 
sparsely). In addition, there is a shared objective term (the global image) that describes the 
overall goal. Imitating this concept, we name the following SOS interpretation as ’’sharing the 
disagreement”. This approach, reported in [37], reduces the local-global gap by encouraging 
the overlapping patches to reach an agreement before they merge their forces by the averaging. 

The proposed boosting algorithm reduces the local-global gap in the following way. Per 
each patch, we define the difference between the local (intermediate) result and the patch 
from the global outcome as a ’’disagreement”. Since each patch is denoised independently. 


^Note that in our terminology, even methods like BM3D [13] are considered as local, even though they share 
information between groups of patches. Indeed, our discussion covers this and related methods as well. In a 
way, the approach taken in [35] offers some sort of remedy to the BM3D method. 
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such disagreement is almost always non-zero and even substantial. Sharing the information 
between the overlapping patches is done by subtracting the disagreement from the noisy image 
patches, i.e., seeking for an agreement between them. These modified patches are the new 
inputs to the denoising algorithm. In this way we push the overlapping patches to share their 
local results, influence each other and reduce the local-global gap. 

More specifically, given an initial denoised version of y and its intermediate patch results, 
we suggest repeating the following procedure: (i) compute the disagreement per each patch, 
(ii) subtract the result from the noisy input patches, (iii) apply the denoising algorithm to 
these modified patches, and (iv) reconstruct the image by averaging on the overlaps. Focusing 
on the K-SVD image denoising, this procedure is detailed in Algorithm 1. 


Algorithm 1 : Sharing the disagreement approach [37]. 

Initialization: 

1: G - initial dictionary. 

2 : Set A: = 0. 

3: Set q? = 0, where G is a ’’disagreement” patch, corresponding to the patch in 
the image. 

Repeat 

1 : Sparse-Coding and Dictionary Update: Solve 


(4.1) 






N 

y]7illQ!i||o + IlDa* - (Riy-qf)lli. 

D,K}f=i 


In practice we approximate the representation of R^y — using the OMP [33] and update 
the previous dictionary using the K-SVD algorithm [1], 

2 : Image Reconstruction: Solve 


(4.2) 


^k+i 


= min 

Z 




- 


l|2 

Il2- 


This term leads to a simple averaging of the denoised patches on the overlaps. 


3: Disagreement Computation: Update 

(4.3) qG^ = - Rix'^+\ 


where qG^ i® the ’’disagreement” between the independent estimation D^+^aG^ the 
corresponding patch from the global outcome RiX^+^. 

Until 

Maximum denoising quality has been reached, else increment k and return to ’’Sparse- 
Coding and Dictionary Update”. 

Output 

X* - the last iteration result. 
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The modified input patches contain their neighbors information, thus encouraging the 
locally denoised patches to agree on the global result. Substituting in 

Equation (4.1) leads to 


(4.4) 


D‘+',{a‘+‘}£, 


N 

min y'7i||«i||o + ||D«i - (R-iY - + RiX^)||i 


Now, by denoting the local residual (method-noise) as r^ = R^y — we get 

N 


(4.5) 


= min y^7i||ai||o + UDa, - (RjX^ + ri)|||, 


where the representation Dcx* is the denoised version of the patch RjX^ + rj. In this formu- 
lation, the input to the K-SVD is a patch from the global (previous iteration) cleaned image 
R^x^, contaminated by its own local method-noise r^. Notice the major differences between 
Equation (1.2) that denoises the method-noise. Equation (1.3) that adds the method-noise to 
the noisy image and then denoises the result, and our local approach that aims at recovering 
the previous global estimation, thereby leading to an agreement between the patches. Our 
algorithm is also different from the EPLL [53], which denoises the previous cleaned image 
without considering its method-noise. 

Still in the context of the K-SVD, Appendix C shows, under some assumptions, an equiv¬ 
alence between the SOS recursive function (Equation (1.4)) and the above ’’sharing the dis¬ 
agreement” algorithm. It is important to emphasize that the former treats the K-SVD as a 
”black-box”, thereby being blind to the K-SVD intermediate results (the independent denoised 
patches, before the patch averaging step). On the contrary, in the case of the disagreement 
approach, these intermediate results are crucial - they are central in the algorithm. Therefore, 
the connection between the SOS and the disagreement algorithms is far from trivial. 

5. Graph Laplacian Interpretation. In this section we present a graph-based analysis to 
the SOS boosting. We start by providing a brief background on graph representation of an 
image in the context of denoising. Second, we explore the graph Laplacian regularization in 
general, and in the context of Equation (3.16), the steady-state outcome of the SOS boosting. 
Finally, we suggest novel recursive algorithms (that treat the denoiser as a ”black-box”) to 
the graph Laplacian regularizers that are described in [16, 2, 23, 24]. 

Recent works [19, 20, 16, 2, 41, 29, 21, 23, 24] suggest representing an image as a weighted 
graph G = {V,E,K), where the vertices V represent the image pixels, the edges E C 
V X V represent the connection/similarity between pairs of pixels, with a corresponding 
weight 

A constructive approach for composing a graph Laplacian for an image is via image de¬ 
noising algorithms. Given a denoising process for an image, which can be represented as a 
matrix multiplication, x = Wy, one can refer to the entry (i, j) as revealing information about 
the proximity between the i-th and j-th pixels. We note that the existence of the matrix W 
does not imply that the denoising process is linear. Rather, the non-linearity is hidden within 
the construction of the entries of W. For example, in the case of the NLM [6], Bilateral [47] 
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and LARK [11] filters, the entries of K can be expressed by 

(5.1) K{i,j) = exp , 

where d{i^j) measures the distance between the (i, j) pixels (or patches), and /i is a smoothing 
parameter. Notice that in the case of the sparsity-based K-SVD denoising [15], the weights 
as defined in Equation (2.8), measure the similarity between the (i, j) pixels through 
the dictionary D. Dealing with an undirected graph the degree di of the vertex Vi can be 
defined by 

(5.2) D{i,i) ^ di ^Y^K{i,j), 

3 

where di is a sum over the weights on the edges that are connected to and £> is a diagonal 
matrix (called the degree matrix), containing the values of in its diagonal^. 

The graph Laplacian has a major importance in describing functions on a graph [50], and 
in the case of image denoising - representing the structure of the underlying signal [16, 2, 29, 
21, 24]. There are several definitions of the graph Laplacian. In the context of the proposed 
SOS boosting, we shall use a normalized Laplacian, defined as 

(5.3) /: = I-W, 

where W is a filter matrix, representing the denoiser /(•) (see Equation (2.8)). Note that W 
is a normalized version of the similarity matrix thus has eigenvalues in a range of 0 to 1. 
There are several ways to obtain W from e.g., W = D~^K is used in [44] and in this 
work (leading to a random walk Laplacian), another way is W = as used in 

[30]. Recently, Kheradmand and Milanfar [24] suggest W = ^ where C is the 

outcome of Sinkhorn algorithm [25]. Notice that different versions of W result in different 
properties of C (refer to [24] for more information). 

In general, the spectrum of a graph is defined by the eigenvectors and eigenvalues of C. In 
the context of image denoising, as argued in [18, 30, 21, 24], the eigenvectors that correspond 
to the small eigenvalues of C encapsulate the underlying structure of the image. On the other 
hand, the eigenvectors that correspond to the large eigenvalues mostly represent the noise. 
Meyer et al. [30] showed that the small eigenvalues are stable even for high noise scenarios. 
As a result, the graph Laplacian can be used as a regularizer, preserving the geometry of the 
image by encouraging similar pixels to remain similar in the final estimate [16, 2]. 

What can we do with Cl The obvious usage of it is as an image adaptive regularizer in 
inverse problems. There are several ways to integrate £ in a cost function, for example, [16, 2] 
suggest solving the following minimization problem^ 

(5.4) X = min llx — yll^ + px^>Cx, 

X 

^The K-SVD degree matrix, as defined in Equation (2.8), also holds the relation described in Equation (5.2). 
According to Theorem 2.1, p is an eigenvector of W = D~^K, corresponding to eigenvalue A = 1, leading to 
D~^K1 = 1. Multiplying both sides by D results in the desired relation K1 — Dl^ i.e., D{i^ i) — . K(i^j). 

^The work in [21] is closely related, but their regularization term is ||Lx|| 2 , and thus it leads to L^L in the 
steady-state formula, where L = E) — iT is an un-normalized graph Laplacian. Thus, we omit it from the next 
discussion. 
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leading to a closed-form expression for x, 

(5.5) x = (I + p/:)“V. 

The authors of [24] suggest an iterative graph-based framework for image restoration. Specif¬ 
ically, in the case of image denoising [23], they suggest a variant to Equation (5.4), 

(5.6) X = min (x — y)^W(x — y) -h px^>Cx. 

X 

Differently from Equation (5.4), the above expression offers a weighted data fidelity term, 
resulting in the following closed-form expression to the final estimate: 

(5.7) X = (W + p/:)“ Wy. 

It turns out that Equation (3.16), the steady-state result of the SOS boosting, i.e., 

(5.8) X* = (I + p(I-W*))-^W*y 

= {1 +pC,)-^ W,y, 

can be also treated as emerging from a graph Laplacian regularizer, being the outcome of the 
following cost function 

(5.9) X* = min llx — W^yll^ + 

X 

Notice the differences between Equations (5.4), (5.6), and (5.9). The last expression suggests 
that SOS aims to find an image that is close to the estimated image W^^y, rather than the 
noisy y itself. In the spirit of the SOS boosting, 

= / (y + - px^ 

we can suggest expressing the above-mentioned graph Laplacian regularization methods, i.e.. 
Equations (5.5) and (5.7), as recursive, providing novel ”black-box” iterative algorithms that 
minimize their corresponding penalty functions without explicitly building the matrix W. 
Starting with Equation (5.5), the steady-state outcome should satisfy 

(5.10) (I + p(I-W))x = y. 

There are many ways to rearrange this expression using the fixed point strategy, in order to 
get a recursive update formula. We shall adopt a path that leads to an iterative process that 
operates on the strengthened image, y-hx^, in order to expose the similarities and differences 
to our scheme. Therefore, we suggest adding Wy — Wy to the RHS, i.e., 

(5.11) X -h px — pWx = y + Wy — Wy. 


Rearranging the above expression results in 


(5.12) 


(1 + P) 


[W (y -b p±) -b (y - Wy)]. 
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As a consequence, the obtained iterative ” black-box” formulation to the conventional graph 
Laplacian regularization [16, 2] is given by 


(5.13) 


x^+i = 


1 

(1 + P) 


/ (y + px'^)+(y-/(y)) 


As can be seen, we got an iterative algorithm that, similar to SOS, operates on the strength¬ 
ened image. However, rather than simply subtracting from the outcome, we add the 
method noise, and then normalize. 

In a similar way. Equation (5.7), which is formulated as 


(5.14) 


(W + p(I- W))x = Wy, 


can be expressed by 

(5.15) X = lw(/9x + y - x), 

P 

and in the general case, the ” black-box” version of [23] is formulated by 

(5.16) x^+i = l/(px^ + y - x^) = -^f{y + p'x"), 

p p +1 

where p' = p—1. Again, we see a close resemblance to our SOS method. However, instead of 
subtracting px^ from the denoised strengthened image, we simply normalize accordingly. 

Equations (5.13) and (5.16) offer two iterative algorithms that are essentially minimizing 
the penalty functions (5.4) and (5.6), respectively. However, these algorithms offer far more 
- both can be applied with the denoiser as a ”black-box”, implying that no explicit matrix 
construction of W (nor C) is required. Furthermore, these schemes, given in the form of 
denoising on the strengthened image, imply that parameter setting is trivial - the noise level 
is nearly cr, regardless of the iteration number. Lastly, an update of W within the iterations 
of these recursive formulas seems most natural. 



(a) Foreman (b) Lena (c) House (d) Fingerprint (e) Peppers 


Figure 2. Visualization of the test images. 

6. Experimental Results. In this section, we provide detailed results of the SOS boosting 
and its local-global variant - ’’sharing the disagreement”. The results are presented for the 
images Foreman, Lena, House, Fingerprint and Peppers (see Figure 2). These images are exten¬ 
sively tested in earlier work, thus enabling a convenient and fair demonstration of the potential 
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of the proposed boosting. The images are corrupted by an additive zero-mean Gaussian noise 
with a standard-deviation a. The denoising performance is evaluated using the Peak Signal 
to Noise Ratio (PSNR), defined as 201ogxo(-^;^|g), where MSE is the Mean Squared Error 
between the original image and its denoised version. 

6.1. SOS Boosting with state-of-the-art algorithms. The proposed SOS boosting is 
applicable to a wide range of denoising algorithms. We demonstrate its abilities by improving 
several state-of-the-art methods: (i) K-SVD [15], (ii) NLM [6, 7], (hi) BM3D [13], and (iv) 
EPLL [53]. The K-SVD [15], which was discussed in detail in this paper, is based on an 
adaptive sparsity model. The NLM [6] leverages the ’’self-similarity” property of natural 
images, i.e., the assumption that each patch may have similar patches within the image. 
The BM3D [13] combines the ’’self-similarity” property with a sparsity model, achieving the 
best restoration and even touches some recently developed image denoising bounds [28]. The 
EPLL [53], which was described in Section 1, represents the image patches using the Gaussian 
Mixture Model (GMM), and encourages the global result to comply with the local patches 
prior. As can be inferred, these algorithms are diverse and build upon different models and 
forces. Furthermore, the EPLL can be considered as a boosting method by-itself, designed to 
improve a GMM denoising algorithm. The diversity of the above algorithms emphasizes the 
potential of the SOS boosting. 

The improved denoising performance is gained simply by applying the authors’ original 
software as a ”black-box”, without any internal algorithmic modifications or parameters set¬ 
tings^. Such modifications may lead to better results and we leave these for future study. In 
order to apply SOS boosting we need to set the parameters p, r, and a modified noise-level 
a (although a is known). The parameter d, which might be a little higher than a, represents 
the noise-level of y-hpx^. We can estimate a automatically (e.g using [52]) or tunning a fixed 
value manually. In the following experiments we choose the second option. We set r = 1 (the 
effect of T* is demonstrated later on) and run several tests to tune p and a per each noise 
level and denoising algorithm, as detailed in Table 1 under the ’SOS params’ column. 

In the case of the EPLL and BM3D, the authors’ software is designed to denoise an input 
image in the range of 0 to 1. As such, we apply the SOS boosting (r = 1) in the following 
formulation: 

(6.1) = Y^p-f(p-p)y + p*") - 

with a corresponding a. In order to remain consistent with the SOS parameters of the K-SVD 
and NLM, which apply Equation (3.14), we provide hereafter the parameters p = and 
d = for the EPLL and BM3D. 

i-p 

Table 1 lists the restoration results of various denoising algorithms and their SOS versions. 
The PSNR values that appear in the ’Orig’ column are obtained by applying the denoising 
algorithm on y using the input noise level a (and not d as done at the consecutive SOS-steps). 
These are also the first estimates of the SOS boosting (i.e., x^). In the case of the K-SVD 

®The original K-SVD uses 8x8 patches, but our experiments show that 9x9 yields nearly the same results 
for the core algorithm, while enabling better improvement with the SOS boosting. As a consequence, in the 
following experiments we demonstrate the results of the 9x9 version. 
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Table 1 

Comparison between the denoising results [PSNR] of various algorithms (K-SVD [15], NLM [7], BM3D 
[13] and EPLL [53]) and their SOS boosting outeomes. Per eaeh denoising algorithm, we apply the authors’ 
original software with the SOS formulation (using r = 1, with the appropriate p and a). The best results per 
eaeh denoising algorithm, image, and noise level are highlighted. 


K-SVD [15 



SOS 

params 

Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 


P 

a 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Imprv. 

10 

0.30 

l.OOcr 

36.92 

37.13 

35.47 

35.58 

36.25 

36.49 

32.27 

32.35 

34.68 

34.71 

35.12 

35.25 

0.13 

20 

0.60 

l.OOcr 

33.81 

34.11 

32.43 

32.67 

33.34 

33.62 

28.31 

28.54 

32.29 

32.35 

32.04 

32.26 

0.22 

25 

1.00 

1.00a 

32.83 

33.12 

31.32 

31.62 

32.39 

32.72 

27.13 

27.44 

31.43 

31.49 

31.02 

31.28 

0.26 

50 

1.00 

1.00a 

28.88 

29.85 

27.75 

28.37 

28.01 

28.98 

23.20 

23.98 

28.16 

28.66 

27.20 

27.97 

0.77 

75 

1.00 

1.00a 

26.24 

27.32 

25.74 

26.40 

25.23 

26.85 

19.93 

21.88 

25.73 

26.72 

24.57 

25.83 

1.26 

100 

1.00 

1.00a 

25.21 

25.39 

24.50 

24.99 

23.69 

24.59 

17.98 

19.61 

24.17 

25.03 

23.11 

23.92 

0.81 

NLM [7] 


SOS 

params 

Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 


P 

a 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Imprv. 

10 

0.10 

1.20a 

35.55 

36.13 

34.32 

34.72 

34.93 

35.39 

31.04 

31.45 

34.02 

34.37 

33.97 

34.41 

0.44 

20 

0.10 

1.10a 

32.78 

33.15 

31.59 

31.84 

32.40 

32.86 

27.26 

27.55 

31.49 

31.78 

31.10 

31.44 

0.34 

25 

0.40 

1.10a 

31.26 

31.88 

30.51 

30.88 

31.22 

31.87 

26.20 

26.22 

30.47 

30.85 

29.93 

30.34 

0.41 

50 

0.50 

1.05a 

27.62 

28.05 

27.31 

27.57 

27.42 

28.00 

23.00 

23.06 

26.79 

26.97 

26.43 

26.73 

0.30 

75 

0.60 

1.05a 

25.38 

26.06 

25.12 

25.75 

24.59 

25.49 

20.84 

21.13 

24.63 

24.94 

24.11 

24.67 

0.56 

100 

0.60 

1.05a 

23.82 

24.21 

23.71 

24.17 

23.07 

23.45 

19.50 

19.67 

23.27 

23.65 

22.67 

23.03 

0.36 

BM3D [13 



SOS 

params 

Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 


P 

a 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Imprv. 

10 

0.05 

1.02a 

37.23 

37.24 

35.84 

35.85 

36.54 

36.55 

32.46 

32.47 

34.96 

34.96 

35.40 

35.41 

0.01 

20 

0.11 

1.03a 

34.50 

34.55 

33.00 

33.02 

33.81 

33.81 

28.82 

28.83 

32.67 

32.68 

32.56 

32.58 

0.02 

25 

0.18 

1.04a 

33.41 

33.48 

32.02 

32.04 

32.90 

32.90 

27.72 

27.72 

31.87 

31.89 

31.58 

31.61 

0.03 

50 

0.25 

1.04a 

30.22 

30.36 

28.98 

29.00 

29.68 

29.80 

24.57 

24.59 

29.09 

29.14 

28.51 

28.58 

0.07 

75 

0.43 

1.04a 

28.09 

28.30 

27.15 

27.21 

27.73 

27.95 

22.84 

22.88 

27.09 

27.11 

26.58 

26.69 

0.11 

100 

0.43 

1.11a 

26.16 

26.42 

25.77 

25.82 

25.74 

25.93 

21.56 

21.67 

25.72 

25.81 

24.99 

25.13 

0.14 

EPLL [ 53 ; 



SOS 

params 

Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 


P 

a 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Imprv. 

10 

0.09 

1.11a 

36.98 

37.09 

35.53 

35.66 

35.67 

35.73 

32.12 

32.33 

34.82 

34.93 

35.02 

35.15 

0.13 

20 

0.09 

1.11a 

33.70 

34.03 

32.57 

32.77 

33.06 

33.33 

28.26 

28.49 

32.48 

32.69 

32.01 

32.26 

0.25 

25 

0.18 

1.11a 

32.44 

32.78 

31.62 

31.84 

32.07 

32.38 

27.14 

27.30 

31.59 

31.87 

30.97 

31.23 

0.26 

50 

0.43 

1.11a 

29.24 

29.60 

28.39 

28.66 

28.78 

29.24 

23.63 

23.69 

28.67 

29.00 

27.74 

28.04 

0.30 

75 

0.43 

1.11a 

27.17 

27.55 

26.53 

26.85 

26.78 

27.28 

21.51 

21.54 

26.73 

27.10 

25.74 

26.06 

0.32 

100 

0.43 

1.11a 

25.58 

25.91 

25.23 

25.49 

25.08 

25.47 

19.77 

19.77 

25.36 

25.73 

24.20 

24.47 

0.27 
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(a) Noisy Image (b) KSVD, 31.20 (c) NLM, 30.02 (d) BM3D, 31.88 (e) EPLL, 30.88 



(f) Algo. 1, 31.85 (g) SOS KSVD,31.91 (h) SOS NLM, 30.56 (i) SOS BM3D, 31.94 (j) SOS EPLL, 31.15 


Figure 3. Visual and PSNR comparisons between standard denoising and boosting outcomes of a 100 x 120 
cropped region from noisy image House (a = 25^. 


denoising [15], at the first SOS-step we apply 20 iterations of sparse-coding and dictionary- 
update, while at the rest SOS-steps we apply only 2 such iterations (we found this to be 
a convenient compromise between runtime and performance). We operate the K-SVD [15], 
NLM [7], BM3D [13] and EPLL [53] for 30, 2, 3 and 4 SOS-steps, respectively. 

The ’average imprv.’ column in Table 1 indicates that the SOS boosting achieves an 
improvement over the original denoising algorithms. More specifically, for all denoising algo¬ 
rithms, images, and noise levels, the SOS outcomes are at least as good as the original results 
and more important - usually better (in terms of PSNR). A clear improvement over the whole 
range of noise levels is achieved for the K-SVD [15], NLM [7, 6] and EPLL [53]. While in the 
case of the BM3D [13], we succeed in improving it slightly, mainly for high noise energy. The 
fact that the BM3D performance is very close to the denoising bound posed in [28] explains 
the difficulties in improving it. A visual comparison is given in Figures 3, 4, and 5 illustrating 
the effectiveness of the SOS boosting. Compared to the original results, the SOS offers better 
restoration of edges (in the case of the K-SVD - focus on the house’s roof, foreman’s eye and 
ear, and lena’s hat). In addition, the SOS obtains cleaner estimations (when using the NLM), 
and less artifacts (for the EPLL and somewhat also for the BM3D). 

In the context of the K-SVD denoising, based on Equation (3.25), we demonstrate the 
effect of T* on the SOS recursive function. Note that we do not test its influence on the 
other denoising algorithms because the information about their eigenvalues range, which is 
required in Equation (3.25), is not derived. Figure 6 plots the average PSNR over the test 
images {a = 50), as a function of the SOS-step, for 3 different parameter settings: First, as a 
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(a) Noisy Image (b) KSVD, 33.72 (c) NLM, 31.64 (d) BM3D, 34.66 (e) EPLL, 33.62 



(f) Algo. 1, 34.37 (g) SOS KSVD,34.38 (h) SOS NLM, 32.28 (i) SOS BM3D, 34.71 (j) SOS EPLL, 34.09 


Figure 4. Visual and PSNR comparisons between standard denoising and boosting outcomes of a 100 x 150 
cropped region from noisy image Foreman (a = 25^. 


baseline, we apply the SOS with p — 1 and r — 1 (without using the closed-form expression 
for T*). Second, we improve the convergence rate by using r* with the same signal-emphasis 
factor {p — 1). Third, we plot the PSNR that obtained by the couple that leads to the best 
restoration {p — 1.1 with the corresponding r*). As a reminder, according to Section 3.3 
and Appendix D, the parameters p and r affect the conditions for convergence and its rate. 
More specifically, a modification of p without an adjustment of r may violate the condition 
for convergence (e.g. according to condition (3.22), the couple p = 1.1 and r = 1 results 
in 7 > 1). Therefore, using r* enables to modify p and still converge, even with the fastest 
rate. These results are consistent with Table 2, which lists the achieved PSNR when applying 
the SOS for 30 steps using the best p and r* (per noise level). As can be seen, r* not only 
results in a faster convergence, but also allows a stronger emphasis of the estimated signal, 
thus leading to better restoration. 

6.2. Sharing the disagreement. We demonstrate the effectiveness of the local-global in¬ 
terpretation of the SOS boosting, which was described in Section 4. The denoising results 
of Table 3 are obtained by applying Algorithm 1 for 30 steps, where each step includes 2 
sparse-coding and dictionary-update iterations. The initial dictionary is obtained by applying 








22 


YANIV ROMANO AND MICHAEL ELAD 



(a) Noisy Image (b) KSVD, 33.06 (c) NLM, 32.13 (d) BM3D, 34.03 (e) EPLL, 33.29 



(f) Algo. 1, 33.56 (g) SOS KSVD,33.48 (h) SOS NLM, 32.41 (i) SOS BM3D, 34.07 (j) SOS EPLL, 33.56 


Figure 5. Visual and PSNR comparisons between standard denoising and boosting outcomes of a 100 x 120 
cropped region from noisy image Lena (a = 20^. 



Figure 6. Demonstration of the effect of t* on the SOS boosting outcome for the K-SVD denoising (a — 50/ 


20 iterations of the K-SVD algorithm. Similarly to the SOS boosting, we tune the parameter 
G per each input a (this variant is limited to p = 1 and r — 1). 

According to Table 3, for all images and noise levels, ’’sharing the disagreement” boosting 
achieves a clear improvement over the original K-SVD algorithm [15]. Notice the resemblance 
and the differences in the PSNR values between Table 3 and the K-SVD part in Table 1. 
In general, the differences originate from the non-linearity of the denoising algorithm - the 
input patch to the sparse-coding step is different between the SOS and its local-global variant. 
As a reminder, the equivalence between these two approaches is valid under the assumption 
of a fixed filter-matrix (see Appendix C for more details). Furthermore, in the case of SOS 
boosting, more freedom is obtained by tuning the parameters p and r, which may lead to better 
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Table 2 

Denoising results [PSNR] of the K-SVD [15] and its SOS boosting outeomes, where we use the parameter 
r* (aeeording to Equation (3.25)), along with the appropriate p and a. The best results per eaeh image and 
noise level are highlighted. 



SOS 

params 

Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 


p 

a 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Orig 

SOS 

Imprv. 

10 

0.30 

l.OOcr 

36.92 

37.14 

35.47 

35.58 

36.25 

36.49 

32.27 

32.35 

34.68 

34.72 

35.12 

35.26 

0.14 

20 

0.60 

1.00a 

33.81 

34.11 

32.43 

32.68 

33.34 

33.62 

28.31 

28.54 

32.29 

32.35 

32.04 

32.26 

0.22 

25 

1.00 

1.00a 

32.83 

33.12 

31.32 

31.65 

32.39 

32.74 

27.13 

27.46 

31.43 

31.53 

31.02 

31.30 

0.28 

50 

1.10 

1.00a 

28.88 

29.86 

27.75 

28.42 

28.01 

29.05 

23.20 

24.03 

28.16 

28.68 

27.20 

28.01 

0.81 

75 

1.20 

1.00a 

26.24 

27.36 

25.74 

26.50 

25.23 

27.08 

19.93 

22.02 

25.73 

26.80 

24.57 

25.95 

1.38 

100 

1.20 

1.00a 

25.21 

25.46 

24.50 

25.09 

23.69 

24.70 

17.98 

19.93 

24.17 

25.17 

23.11 

24.07 

0.96 


Table 3 

Comparison between the denoising results [PSNR] of the original K-SVD algorithm [15] and its ”sharing the 
disagreement” boosting outeome (Algorithm 1). The best results per eaeh image and noise level are highlighted. 




Foreman 

Lena 

House 

Fingerprint 

Peppers 

Average 



Orig 

Boost 

Orig 

Boost 

Orig 

Boost 

Orig 

Boost 

Orig 

Boost 

Orig 

Boost 

Imprv. 

10 

1.08a 

36.92 

37.13 

35.47 

35.58 

36.25 

36.34 

32.27 

32.35 

34.68 

34.70 

35.12 

35.22 

0.10 

20 

1.02a 

33.81 

34.11 

32.43 

32.68 

33.34 

33.56 

28.31 

28.59 

32.29 

32.37 

32.04 

32.26 

0.22 

25 

1.02a 

32.83 

33.17 

31.32 

31.64 

32.39 

32.71 

27.13 

27.47 

31.43 

31.60 

31.02 

31.32 

0.30 

50 

1.00a 

28.88 

29.37 

27.75 

28.28 

28.01 

28.67 

23.20 

24.04 

28.16 

28.55 

27.20 

27.78 

0.58 

75 

1.00a 

26.24 

27.04 

25.74 

26.28 

25.23 

26.54 

19.93 

21.76 

25.73 

26.52 

24.57 

25.63 

1.06 

100 

1.00a 

25.21 

25.28 

24.50 

24.91 

23.69 

24.43 

17.98 

19.82 

24.17 

24.92 

23.11 

23.87 

0.76 


utilization of the prior (as shown in Figure 6 and Table 2). However, visually, according to 
Figure 3, 4, and 5, the outcomes of the SOS and its local-global variant are very similar, both 
of them improve effectively the restoration of the underlying signal. 

To conclude, we demonstrate the potential of the SOS-boosting and its local-global in¬ 
terpretation. The proposed algorithm achieves a clear and meaningful improvement over the 
examined state-of-the-art denoising algorithms, both visually in terms of PSNR. 

7. Conclusions and Future Directions. We have presented the SOS boosting - a generic 
method for improving various image denoising algorithms. The improvement is achieved by 
treating the denoiser as a ”black-box” and repeating 3 simple SOS-steps: (i) Strengthening the 
signal, (ii) Operating the denoising algorithm, and (hi) Subtraeting the previous denoised im¬ 
age from the result. In addition, we provided an interesting local-global interpretation, called 
’’sharing the disagreement” boosting, indicating that the SOS boosting not only leverages the 
improved SNR of the estimates, but also reduces the gap between the local patch processing 
and the global need for a whole denoised image, all in the context of the K-SVD denoising al¬ 
gorithm. We also constructed the matrix-formulation of the K-SVD (and similar algorithms), 
showing that its eigenvalues are in the range of 0 to 1. Under these conditions, we have stud¬ 
ied the convergence of the SOS boosting recursive function, leading to the conclusion that for 
various known denoising algorithms, the SOS boosting is guaranteed to converge. Moreover, 
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a generalization of the SOS function has been obtained by introducing two parameters that 
govern the steady-state result, soften the requirements for convergence (the eigenvalues range) 
and the rate-of-convergence. We also provided a closed-form expression for the parameter that 
leads to the fastest convergence. 

Finally, we have introduced a graph-based interpretation, showing that the SOS boosting 
acts as a graph Laplacian regularization method, thus effectively estimating the structure of 
the underlying signal. Inspired by the SOS scheme, we suggested novel recursive algorithms 
that treat the denoiser as a ”black-box” in order to minimize related graph Laplacian objective 
functions, without explicitly constructing the weighted graph. 

The proposed algorithm is easy to use, it reduces the local-global gap, acting as a graph 
Laplacian regularizer, it is applicable to a wide range of denoising algorithms and it converges 
- these make it a powerful and convenient tool for improving various denoising algorithms, as 
demonstrated in the experiments. 

It is intriguing to study the proposed iterative algorithms that are defined in Section 5 
(which minimize different graph Laplacian cost functions). They may lead to better results 
than the original methods due to the non-linearity of the denoiser and its adaptivity to the 
signal-strengthened image. We hope that other restoration problems, such as super-resolution 
[34], interpolation/inpainting [38] and more, could also benefit from a similar concept. 

Appendix A. Periodic Boundary Condition. Following Equation (2.8), the periodic 
boundary condition affects the term which is a diagonal matrix that counts 

the number of appearances per pixel in the final patch-averaging. Due to boundary effects, 
the values along the diagonal in this matrix are different (since the number of overlapping 
patches in the image borders is smaller than in other areas). As shown in Appendix B, the 
numerator of Equation (2.8) is a symmetric and positive definite matrix. Each row of this 
matrix is normalized by the number of overlapping patches, i.e., by the corresponding di¬ 
agonal element from (/il -h RfRi). As a result, the rows and columns are normalized 
by different constants which ruin the symmetric property. By assuming periodic boundary 
condition, all the pixels have the same number of representations, which equals to the patch 
size. In this case, we get where n is the patch size and thus, the rows and 

columns are normalized by the same constant, which preserves the symmetric property of W. 

Appendix B. Properties of the K-SVD Filter-Matrix. Proof of property 1: symmetric 
W = W^. Following Equation (2.8) and based on the assumption of periodic boundary 
condition (see Appendix A) the matrix W can be expressed as 


(B.l) 
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Notice that the term = R^D 5 .(Dj.D 5 j is symmetric (in fact, it is also positive 

semi-definite (PSD), as it is built of RJ"Z^R^, where Z^ is a projection matrix [22]). Thus, 
W is built as a sum of + 1 matrices, each of them symmetric, which leads to the claimed 
symmetry, W = W^. ■ 

Proof of properties 2 & 3: positive definite W >- 0 and A^^^(W) > As we have seen 
above, W can be written as 


(B.2) 


W= —^1 +y fRfD,, (D^D,.) 


II 


i=l 

N 


-1 + 


—yz 

-I- n 


a-\- n a-\- n 

i=l 

= bl + A, 


where A = Zz, and b = > 0. As mentioned above, Z^ is PSD and therefore, 

according to Equation (B.2), A is a linear combination of Z^ ^ 0, thus it is PSD as well [22]. 
Finally, based on the fact that the eigenvalues of A + bl are lower-bounded by b > 0, we get 
that W y 0, with minimal eigenvalue satisfying 

(B.3) A^i„(W) > b = m 

ja + n 

Proof of property 4 (& 5): Wj, = = 1. This property originates directly from the 

K-SVD denoising algorithm, which preserves the DC component of the image. In general, the 
trained dictionary is adapted to the image patches after their DC is removed. Once trained, 
the DC is returned as an additional atom do- Thus, this DC atom is necessarily orthogonal 
to the rest of the dictionary atoms. Each patch is represented by 

(B.4) D^, = [do,G?i,rf2,...] = [rfo,D5,], 

where do ^ is the DC atom (the DC atom is included if the mean of the patch is not zero) 
and for the rest of the atoms (if any), dfdo = 0, i.e., dg D^. = 0. Note that the Gram matrix 
in this case is block-diagonal 


(B.5) 


dJd, 

So 


1 0 
0 


Following Equation (B.2), when multiplying W by a constant image, 1., we get 


(B.6) 


W1 = 




N 


-1 + 


II + n n + n 


EZ'i- 


i=l 


Let us look at the term Z^l, 


(B.7) 
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(c) Q3 (d) Q4 


Figure 7. Dividing r x c = 6 x 6 input image (its pixels are numbered in 1 , 2 , 36 ^ into ^Jn x ^Jn = 2x2 
overlapping patehes (solid squares in different eolors). We assume a eyeling proeessing of the patehes (periodie 
boundary eondition), thus there are iV = 36 sueh patehes, whieh ean be divided into possible distinet 

groups of non-overlapping patehes: (a) - without any shift, (b) D2 - down shift, (e) Q3 - right shift, (d) $24 

- down & right shift. 


R^l = 1 - this is a shorter constant vector of length n. D1 = [n,0,0, ...]^ due to the 
orthogonality of the rest of the atoms to the DC. Multiplication of the the inverse of DJ.D^. 
results with [n,0,0, The outcome of D5jn,0,0, ...]^ = ndo = 1 is the desired DC patch 
of length n. Finally, Hf returns the resulting constant patch back to its original location in 
the image. 

Returning to Equation (B.6), the input image 1 is divided into N overlapping patches 
(one per pixel) of length n, where each DC patch is represented perfectly by Z^. Since each 
pixel appears in n patches, we get that = '^1* ^ result, Wl, = = 1 

preserves the DC of the image. Based on the symmetric property of W, we get that = 

W1 = 1. ■ 

Proof of property 6 (& 7): ||W ||2 = 1. In order to denoise the image, we break it into 
^/n X ^yn overlapping patches. The following proof relies on the observation that we can 
divide the N overlapping patches into distinct groups of non-overlapping patches, as 

demonstrated in Figure 7. As a consequence, the matrix W, which is a sum over N projection 
matrices (one per each patch), can be expressed as a sum over n distinct groups: 


(B.8) 




W= ^—1 + 

jj, Tt jJi 




■I + 

fi + n /X + 


i=l 

1 ^ 

—EC 

■f n ^ V 


:RfZ,R, 




i=l 
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-1 + 
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—E 

_ 1 _ rn r 


li + n iJL + n 






-1 + 
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—Ew, 


Li-fn a -\- n 
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Based on the property that = 0 for all (fc ^ /) G rij, the following shows that Wj is 

an idempotent matrix: 


(B.9) 




^ ^ i^k ^/eRfc) 


J2iRfZiRi) 

k^Llj 



^ ^ {^k ^k^k^ 1 

{+k^k^k^ 


keQj 

= ^ (^RfcZ/eZ/eR^ 

kefij 

ke^j 

= w„ 

where we have used the equality Ra^R^ = I G and 

(B.IO) (Zfc)2= (d,, 


-^t^T 
Si 


= D,, (Df D,^)- D 

= ^k- 


As a result, following Equation (B.9), we can infer that ||Wj ||2 = 1 [22]. Finally, using the 
matrix-norm inequalities we get 


(B.ll) 


|W||o = 


M 


-1 + 


1 ” 

—Ew,i 

_ 1 _ ri 


< 


/i + n fi + n 

—^ • ||l||2 +-Xll^ilb 

11 + n 11 + n 

Li n 

+ 


{1 + n fi + n 

= 1 . 


To conclude, based on the above and by relying on the property that 1 is an eigenvalue of W, 
we get that A^accCW) = 1, i.e, | 1 W ||2 = 1. ■ 

Proof of property 8: ||W — I ||2 < In general, the eigenvalues of A + bl are bigger 

than the eigenvalues of A by the constant b [22]. Therefore, the eigenvalues of (W — I) equal 
to A(W) — 1. Based on < A(W) < 1, we get that 

(B.12) ||w-I||2<|A^,,(W)-1| 


/i + n 
n 

fi + n 
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Appendix C. Equivalence between the SOS boosting and sharing the disagreement 
procedure. In the context of the K-SVD image denoising [15], we show an equivalence be¬ 
tween the SOS boosting recursive function (Equation (1.4)) and the disagreement and sharing 
approach (Algorithm 1). The following study assumes fixed supports and dictionary during 
the iterations, i.e., the projection matrix D^. of the patch is known and fixed. In addition, 
a periodic boundary is considered (see Appendix A), and we use the K-SVD matrix form (see 
Equation (B.l)) with /i = 0, i.e.. 


(C.l) 


’W=iERrD.,(DSD.,)-‘DjR. 

” i=l 

1 ^ 

= -y^RfZiRi. 

71 


i=l 


Following Algorithm 1, we denote by the iteration input patch to the denoising 
algorithm, which is influenced by the neighbors information. Using the projection matrix 
Zi = D 5 .(DJ.D 5 J“^D^, let us compute the disagreement patch - the difference between an 
independent denoised patch, as defined in Equation (2.6), 

(C.2) pf = Zipf, 

and its corresponding patch from the global outcome (after patch averaging), R^x^, thus 
expressed by 

(C.3) qf = pf-RiX^. 

Next, we subtract the disagreement patch from the corresponding noisy one, i.e., 

(C.4) ppi = R^y - qf 

= R^y - pf + Rix^ 

= Riy - + Rix^ 

Following Equation (C.2), the denoised version of pf"*"^ is given by 
(C.5) pf+i = Zippi 

= ZjRjy - %ZiPi + ZjRjX^ 

= ZjRjy - Zipf + ZjRjX^ 

= ZiRi (y + x'^) - pf 


where we use the idempotent property of Z^. Similarly to Equation (2.8) and based on 
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Figure 8. Illustration o/0(t, p, A), the eigenvalues of the SOS error’s transition matrix (in absolute value), 
as a funetion of t. The dashed (blue), solid (blaek) and dotted (magenta) lines are eorresponding to <f> with 
the arguments Xmin, Xi and Xmax, respeetively. The horizontal blaek dash-dotted line denotes the eondition for 
eonvergenee, determining the values of and (red eireles). The highlighted line illustrates the funetion 

maxi 0 (t, p, Ai), where r* obtains its minimal value (green eirele). 


Equation (C.l), the global denoised image is formulated by 
(C. 6 ) = 


\k+l 


i=l 

N 


ij3Rf(ZiR.(y + i'=)-p: 

i=l 

N \ N 

\ i=l / i=l 




= W y+ x^ -X 


Thus, the SOS boosting (Equation (1.4)) and the ’’sharing the disagreement” algorithms are 
equivalent for a fixed W. 


Appendix D. Seeking for the fastest convergence. We aim to provide conditions for 
the SOS algorithm to converge in terms of the parameters p and r, and in addition get closed- 
form expression for r*, the solution of Equation (3.23). The eigenvalues of the SOS error’s 
transition matrix (in absolute value) are formulated by 


(D.l) 4>{T,p,Xi) = \T{pXi-p-l) + ll 

where are the eigenvalues of W. In the following analysis we shall assume that Aj < 1. 

Figure 8 plots 0 (t, p, (/)(t, p, A^) and 0 (t, p, A^^^) as a function of r, for p > 0. As can 

been seen, (j) has a negative slope for 0 < r < and a positive slope for r > 

All of these are true under the assumption that pA^ — p — 1 < 0, which always holds for A^ = 1 
and for 

(D.2) p > p^^^ = min -—MXi ^ 1. 

i I — Xi 

Next, we shall find the valid range of r G ( 7 -™^^ satisfying (j) <1. Following Figure 

8, the minimal r that leads to an intersection with 0 = 1 is 
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p 


Figure 9. Demonstration o/maxi ^(t, p, A) for X G [0.1,1], where eold and warm eolors indieate small and 
large values, respeetively. The magenta dash-dotted, dashed and solid lines plot the boundaries for eonvergenee, 
eorresponding to ^rmn respeetively. The white dotted line plots the analytie expression of r* 

as a funetion of p. 


Then, by increasing r we get (/><!, until reaching to - the first r > that obtains 
(j) — 1 again. As demonstrated in Figure 8, for p > 0, we get ^ . While for 


p < 0 we get — 


p+l-pA 

n 


p+l-pA 

n 


thus 


(D.4) 


P T 1 P^min P + 1 P^n 


The obtained conditions on p and r are illustrated in Figure 9, which plots the function 
max^ p, A^) for A G [0.1,1], where (/><!. As can be seen, and bound 

perfectly the valid range for convergence. 

We now turn to discuss r*, the solution of Equation (3.23), 


T* = min max p, AA s.t. Vi p, AA < 1. 

r l<i<N 

Following Figure 8, under the obtained conditions on p and r, we seek for minimum value 
of the highlighted graph (the green circle). As can be seen, r* is obtained by the following 
equality 


(D-5) A(t ,P, Att^^ti) — ^P-i^max^ 

Based on condition (D.2), we can infer that 

(D.6) 0('7-*, P, Aw) = -T*(pAw - p - 1) - 1 

and 

(D-7) 5 P") ^max) ~ 'T {p^max P 1) + 1. 

Substituting Equations (D.6) and (D.7) into Equation (D.5) lead to 


T 


2(p + 1) — p{\min + A max^ 
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In addition, by substituting r* into Equation (D.7) we get 

_ pi^^max ^min) 

2[p + 1 ) — p{Xmin + ^ max) 

An illustration that r* obtains the minimal eigenvalue of the error’s transition matrix is 
shown in Figure 9, as a curve running through all possible values of p. Note that the fastest 
convergence is obtained for the couple r — 1 and p — 0 ( 7 * = 0 ), i.e., applying the original 
denoising algorithm only once, without any SOS step. However, as we aim to improve the 
denoising performance by strengthening the underlying signal (setting p ^ 0 ), this choice of 
parameters is meaningless. 
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